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ABSTRACT 

A procedural variable, the use of computers to score 
the students* assignments, is being studied as it affects performance 
on correspondence courses. The format used is known as CALS-Computer 
Assisted Lesson Service • Courses were converted to an objective type 
lesson format which the student responded to using an electronically 
read answer sheet* The computer scored the lessons and printed 
comments to the student regarding the specific answer selected • 
Compared to the conventional course format, the CALS version 
positively affected lesson submissions, but had no effect on final 
examination scores. The effects of this procedure on completion rates 
have not, as of this time, been assessed. It is presumed that CALo 
will be a viable factor, particularly for identifiable subject matter 
area courses. (Author) 
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Introduction 

The United States Armed Forces Institute (USAFI) offers an extensive 
array of correspondence courses for personnel on active duty in the Army, 
j /^ Navy, Air Force, Marine Corps, or Coast Guard, The courses are at both 
Q J the secondary and college levels and cover occupational as well as academic 
*^ ^' subjects. Last year there were about 84, 000 correspondence enrollments in 
Q ) USAFI courses. Approximately 25,000 lessons are received monthly at USAFI. 

The computer configuration at USAFI consists of an off-line mark sense 

o 

reader and a second generation computer with an on-line printer. What has 
•5T-J come to be known as the Computer Assisted Lesson Service (CALS) represents 

.4 

an attempt to exploit this rather limited computer capability in the grading and 
processing of lessons. The aim of CALS was twofold: (1) To save money. The 
grading of lessons by the computer proved to be considerably less expensive than 
grading by a human instructor. 

(2) To improve service to the student. It was felt that two improvements 
in service could be realized through CALS: 

a. Shorter lesson turnaround time. 

b. More comprehensive feedback to the student. 



In correspondence study, the long delay between lesson preparation and 
instructor response is a serious problem, especially for an institution whose 
students are scattered around the world. Through CALS, lessons are graded 
within one day of reception and mailed back to the student promptly. This com- 
pares with an in-house delay which might otherwise run to several days. 

The paucity of instructor feedback to the student is also a problem in corre- 
spondence study. In grading lessons, it is usually not practical for the instructor 
to take the time necessary to write extensive comments on the paper. Sometimes 
when the instructor does write comments on the margin, the student has difficulty 
reading them. A crucial element in CALS is the consistent and well thought out 
instructor comments which are keyed to student responses. In adapting a course 
to CALS: (1) lesson assignments are converted to objective items — usually 
multiple choice, (2) instructor comments are developed on the basis of an analysis 
of probable student errors, and (3) a computer program is written for keying these 
comments to student response alternatives. 

In the preparation of lesson assignments, the student uses answer sheets 
which can be machine scored. The computer printout returned to him contains a 
lesson grade and instructor comments appropriate to his responses. A few 
examples follow: 

In an introductory course in refrigeration, an item on one lesson 

reads as follows: 



What kind of tubing should be used inside a refrigerator 
cabinet ? 

(a) soft copper 

(b) hard copper 

(c> ^tinnfiH soft nnp per 

(d) tinned hard copper 

The correct answer is (c) "tinned soft copper. " If a student chooses 

alternative (a) "roft copper, " the computer printout returned to him 

contains the following comment in relation to this item: "You are 

partially correct in that you indicate soft copper, but this is not the best 

answer. The tubing should be tinned soft copper. The main reason for 

tinning is ease of cleaning and appearance. " 

In a carpentry course, a lesson item reads as follows: 

To lay out an ellipse for a pipe going through a roof, you must 

know: 

(a) the slope of the roof 

(b) the allowance required for air space around the pipe 

(c) the height the pipe will extend above the roof 

(d) both a and b^ 

The correct answer is (d). 

If a student selects (a), he is told: "This is partly correct. You 
must know the unit rise per foot of run of the roof but you must also know 
the allowance for air space around the pipe." 



If he selects (b), he is told: "This is partly correct. You must 
know the allowance for the air space around the pipe but you also have 
to know the slope of the roof in terms of unit rise per foot. " See 
page of your textbook. 

If he selects (c), he is told: "This is a wrong answer. Whether 
the pipe is to extend one foot or 10 feet above the roof makes no difference 
when laying out the ellipse in the roof. " 

If he selects (d), he is told: "Very good. This is the best answer. " 

In the refrigeration course, the following lesson item appears: 

Which of the following statements is not considered to be one of 
the basic theimal laws?" 

(a) Fluids absorb heat while changing from a liquid state to a 
vapor state, and give up heat in changing from a vapor to 
a liquid. 

(b) The temperature at which a change of state occurs is constant 
during the change, but this temperature will vary with the 
pressure. 

(c) Heat will flow only from a body at a higher temperature to a 
body at a lower temperature (hot to cold. ) 

(d) Refrigerants must be imder a high pressure in order to evaporate 
easily and absorb heat. 

The correct answer is (d). Comments keyed to each response are 
as follows: 

(a) This is a basic thermal law. It takes heat to change water to 
steam, 970.4 BTU/LB at 212F. By the same token, when 
going from steam to water 970.4 BTU/LB at 212 F is given off. 
Check page 24 in your textbook. 



(b) This is a basic thermal law. For example, there is no 
change in temperature of ice when it melts at 32 F if the 
pressure remains constant. Melting ice keeps a temper- 
ature of 32 F until all the ice is melted. Check page 25 
in your textbook. 

•c*) This is a basic thermal law. . 

Stop and think about it and-yoiuMllJiaveJo^gree that 
heat flows only from hot to cold. If you want to add heat 
to an object, it is necessary to have a heat source in 
order for the object to warm. Check and review page 12 
in your textbook. 

(d) You are right, this is not one of the basic thermal laws. 
Refrigerants do not have to be under a high pressure to 
evaporate easily and absorb heat. In fact, the less pressure 
the easier it is for them to evaporate. 

The following lesson item appears in a United States history course: 

Opposition to immediate admission of Texas as a state in 1836 

came primarily from 

(a) Southern cotton-growers 

(b) Western cattle-raisers 

(c) whigs and abolitionists 

(d) Southwestern farmers 

(c) is the correct answer. The following comment is keyed to an 
incorrect response: 

"While Southerners and Westerners generally favored the annexa- 
tion of Texas in 1836, opposition came from Northeastern Whigs and 
abolitionists. Many eastern Whigs feared a war with Mexico if Texas 
were taken into the Union, and also were fearful that Texas as a state 
would enhance Southern political power to block legislation favorable to 
Northern economic interests. The abolitionists opposed annexation on the 
grounds it would bring in a new slave state and open the territories to 
slavery, (p. 275)" 



In addition to the comments keyed to specific student responses, there 
2re more general comments appropriate to the student's performance on the 
lesson. For the student who scores high on his first lesson, it might be: 
"You are off to a good start. Keep up the good work. " The student with a low 
lesson grade might be offered words of encouragement and asked to resubmit 
the lesson. 

— Ev 

The effectiveness of CALS is being assessed in terms of lesson volume, 
number of course completions, and student performance on the final examination. 
In terms of all three criteria, the results are generally favorable. 

Sixteen USAFI courses have been adapted in their entirety to CALS. 
Another 14 courses have been adapted in part. Six of these 30 courses have 
been adapted to CALS so recently that no assessment has yet been done. 
Another 12 courses have been so extensively restructured as to make it difficult 
to assess the effects of CALS per se. In 11 of the remaining 12 CALS courses, 
increases in lesson submission rates have occurred. 

One of the first courses adapted to CALS was a high school general mathe- 
matics course. For a period of several months, the CALS version of the course 
was issued randomly to one group of enroUees and the conventional version to 
another group. That is, if the enroUee's social security account number ended 
with an odd number he was issued the CALS version, otherwise he received the 
conventional version. Statistics on lesson submissions are as follows: (1) The 
mean number of lessons per student in the CALS group is about seven, as com- 
pared to five in the conventional group; (2) Sixty-three percent of the CALS 



students have submitted one or more lessons as compared to 55 percent of the 
others, and (3) Fifteen percent of the CALS students, as compared to 8 percent 
of the others, have submitted all the lesson assignments in the course. 

In terms of mean scores on the final examination for the course, the two 
groups were virtually identical — 51.3 versus 51. !• 



Somewhat similar data could be cited for other courses. The results, 
however, are not uniform across courses. The increase in lesson submissions 
associated with CALS has been substantially greater in some courses than others. 
The one course which has shown virtually no increase in lesson volume associated 
with CALS is an introductory accounting course. The CALS students in this 
course actually had lower start rates. That is, 70 percent of the students en- 
rolled in the conventional course have submitted one or more lessons as compared 
to 64 percent of the CALS group. Moreover, the CALS students seem to have 
more difficulty with subsequent portions of the course. 

The courses in which CALS has shown the most marked effects are three 
high school courses (an English course and two general mathematics courses) 
and two vocational-technical courses. An auto mechanics course and a refriger- 
ation course have shown quite marked increases in lesson submissions following 
their adaptation to CALS. 

How are these effects of CALS to be accounted for? The increase in 
lesson submissions seems likely to be due to one or more of the following: 
(1) the multiple-choice format of lesson assignments, (2) the more prompt re- 
turn of lessons processed through CALS, (3) the greater feedback to the students, 
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and (4) the novelty of CALS, Earlier studies at USAFI have shown that con- 
verting lesson assignments to multiple-choice items does in itself result in 
increased lesson submissions. We have not yet been able to assess the effects 
of the other factors. 

Student Reactions 

In addition to the analysis of data on lesson submissions and examination 
scores, we have also surveyed student reactions to CALS. Several months 
ago, questionnaires were sent to students in four courses — two general mathe- 
matics courses, a criminology course, and a refrigeration course. Reactions 
were somewhat mixed. In response to the statement "Computer printed comments 
on CALS lessons are helpful and to the point, " 75 percent or more of the students 
in all four courses either agreed or strongly agreed. But the statement "Com- 
puter printed comments are more helpful than instructor written comments," 
got agreement from only 12 percent of the criminology students, 25 percent of 
the general math students, and 40 percent of the refrigeration students. The 
item "CALS is another step toward depersonalized instruction, " got 50 percent 
agreement among refrigeration students, 60 percent agreement by general math 
students, and better than 70 percent agreement by criminology students. 
Generally, as these data suggest, students in the criminology course were less 
favorably disposed toward CALS than were the students in the other courses. 
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Summary 

In summary, the effoi^t to improve instructional services to the corre- 
spondence student through utilization of a rather limited computer capability 
has met with at least limited success. While the results are not unmixed, 
they do indicate that some improvement in instructional effectiveness has been 
realized through CALS. 



